BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning

Understanding the global optimality in deep learning (DL) has been attracting more and more attention recently. Conventional DL solvers, however, have not been developed intentionally to seek for such global optimality. In this paper we propose a novel approximation algorithm, BPGrad, towards optimizing deep models globally via branch and pruning. Our BPGrad algorithm is based on the assumption of Lipschitz continuity in DL, and as a result it can adaptively determine the step size for current gradient given the history of previous updates, wherein theoretically no smaller steps can achieve the global optimality. We prove that, by repeating such branch-and-pruning procedure, we can locate the global optimality within finite iterations. Empirically an efficient solver based on BPGrad for DL is proposed as well, and it outperforms conventional DL solvers such as Adagrad, Adadelta, RMSProp, and Adam in the tasks of object recognition, detection, and segmentation

The Ambivalence towards Natural Characteristics in the Interactions with Small Indoor Flying Robots

Nature and humanity have been in constant and varied interactions throughout history and technological epochs. I conjecture that there are benefits to integrating natural characteristics into robot designs for their interactions with humans. I test this conjecture experimentally with a focus on close-range interactions with flying robots, using Research through Design (RtD) and mixed-methods approaches. In my half-PhD seminar, I will discuss the two studies I have carried out, namely: 1) overlaying natural sounds, i.e. birdsong and rain sound, on a noisy flying robot at three proxemic distances (N=56), accepted by ACM THRI journal: https://doi.org/10.1145/3579859; 2) exploring potential usage scenarios of indoor drones (N=66), including investigating the notion of pet drone. In both studies, I found that participants were ambivalent towards the natural characteristics depending on given circumstances. This informs that utilising natural characteristics in human-robot interaction (HRI) may be compelling; however, there are pitfalls, and comprehensive strategies and careful considerations are required. I will also briefly present my research ideas of investigating interactions with bioinspired and biohybrid flying robots for my subsequent studies.Discussion leader: Wendy Ju, Associate Professor at the Jacobs Technion-Cornell Institute at Cornell Tech, USATime: 2023-05-12 Friday 13:00-15:00 CET;Place: Room Windows, 3rd Floor, Building Kuggen, Lindholmsplatsen 1, 417 56 Gothenbur

Compact Digital Predistortion for Multi-band and Wide-band RF Transmitters

This thesis is focusing on developing a compact digital predistortion (DPD) system which costs less DPD added power consumptions. It explores a new theory and techniques to relieve the requirement of the number of training samples and the sampling-rate of feedback ADCs in DPD systems. A new theory about the information carried by training samples is introduced. It connects the generalized error of the DPD estimation algorithm with the statistical properties of modulated signals. Secondly, based on the proposed theory, this work introduces a compressed sample selection method to reduce the number of training samples by only selecting the minimal samples which satisfy the foreknown probability information. The number of training samples and complex multiplication operations required for coefficients estimation can be reduced by more than ten times without additional calculation resource. Thirdly, based on the proposed theory, this thesis proves that theoretically a DPD system using memory polynomial based behavioural modes and least-square (LS) based algorithms can be performed with any sampling-rate of feedback samples. The principle, implementation and practical concerns of the undersampling DPD which uses lower sampling-rate ADC are then introduced. Finally, the observation bandwidth of DPD systems can be extended by the proposed multi-rate track-and-hold circuits with the associated algorithm. By addressing several parameters of ADC and corresponding DPD algorithm, multi-GHz observation bandwidth using only a 61.44MHz ADC is achieved, and demonstrated the satisfactory linearization performance of multi-band and continued wideband RF transmitter applications via extensive experimental tests

Risk factors correlated to lymph node metastasis in thymic epithelial tumors and the prognostic significance of lymph node dissection for thymic carcinomas and thymic neuroendocrine tumors

Thymustumore werden im Allgemeinen als wenig maligne Tumore, mit geringer Wahrscheinlichkeit einer Lymphknotenmetastasierung (LNM) angesehen. Diese Studie zielt darauf ab, die Faktoren, die mit einer möglichen LNM bei Thymusepithelialtumoren (TETs) stehen zusammenhängen , zu analysieren und den Einfluss der Lymphknotendissektionen (LND) bei pathologisch gesicherten Hochrisikotypen (Thymuskarzinomen, TCs und Nuroendokrinen Tumoren des Thymus, TNETs) auf die Prognose zu untersuchen. Diese Studie analysierte systematisch die klinisch-pathologischen Informationen von Pat. mit Thymus Malignomen in der Surveillance, Epidemiology, and End Results Datenbank. Zunächst wurde die Inzidenz von diesen Tumoren zusammengefasst, dann wurden die relevanten klinisch-pathologischen Faktoren von Pat. mit Thymomen (A-B3), Thymuskarzinomen (TCs) und Thymus neuroendokrinen Tumoren (TNETs), die operiert und bei denen Lymphknoten exsipiert wurde, gesammelt. Außerdem wurden unabhängig von der LNM in Beziehung stehende Variablen mittels Logistik-Regression bestimmt. Schließlich wurden Pat., bei denen die diagnostizierten TCs und TNETs chirurgisch behandelt wurden, gesammelt und die Prognose bei unterschiedlichem Lymphknotenstatus analysiert. Die Cox-Analyse wurde verwendet, um die Variablen im Zusammenhang mit der Prognose des Gesamtüberlebens (OS) und des krebsspezifischen Überlebens (CSS) zu analysieren. Propensity Score Matching (PSM) wurde für die Subgruppenanalyse von Pat. mit unterschiedlichem Lymphknotenstatus verwendet. Insgesamt wurden 5934 Pat. mit pathologisch gesicherten Thymus-Malignomen eingeschlossen am häufigsten waren Thzmome gefolgt von TCs und TNETS. Insgesamt wurden 1048 TETs Pat. operiert und erhielten eine LND. Der Gesamtanteil der TETs Pat.. mit LNM betrug 1,1%. Die LNM-Rate bei Thymomen, TCs und TNETs betrug 6,8%, 30,2% bzw. 61,1%. Histologietyp sowie T-Stadium waren unabhängige Faktoren, die mit LNM in der multivariaten Logistikanalyse korrelierten. Es gab 812 Pat. mit TCs und TNETs, die sich einer chirurgischen Behandlung unterzogen hatten, darunter waren 76,7% TCs und 11,6% TNETs. Etwa 398 Pat. erhielten eine LND und von diesen hatten 36,2% eine LNM. In der multivariaten Cox-Analyse von OS und CSS war die Prognose von LND- Pat. signifikant schlechter als die von N0 Pat.. Der prognostische Unterschied zwischen N+ und LND- Pat. war nicht statistisch signifikant. Nach PSM ist in der univariaten Analyse und in der multivariaten Subgruppenanalyse von OS und CSS das Überleben von N0 Pat. immer noch besser als das von LND- und N+ Gruppen, jedoch zeigte der Prognoseunterschied zwischen LND- und N+ Pat. keine statistische Signifikanz in der multivariaten Analyse (P >0.05). Lymphknotenbeteiligung ist bei TETs nicht ungewöhnlich. Hauptfaktoren im Zusammenhang mit LNM in TETs sind der Histologietyp sowie das T-Stadium. LND in TCs und TNETs kann dabei helfen einen genaueren Lymphknotenstatus, sowie die Langzeitprognose von Patienten besser zu beurteilen.Thymic tumors are generally considered with low degree of malignancy, and the probability of lymph node metastasis (LNM) is low. This study aims to further comprehensively analyze the related factors of LNM in thymic epithelial tumors (TETs) and investigate the impact of lymph node dissection (LND) in high-risk pathological types (thymic carcinomas, TCs and thymic neuroendocrine tumors, TNETs) on the prognosis. The present research systematically analyzed the clinicopathological information of patients with thymic malignancies in the Surveillance, Epidemiology, and End Results (SEER) database. The overall incidence of tumors was firstly analyzed. Furtherly, the relevant clinicopathological factors of thymoma (A-B3), thymic carcinomas (TCs), and thymic neuroendocrine tumors (TNETs) who had surgical treatment and underwent ≥1 lymph node examined were collected, and variables independently related to LNM were determined via Logistics regression. Finally, the patients diagnosed TCs and TNETs undergoing operative treatment were collected, and the differences in the prognosis of patients with different lymph node status were analyzed. Univariate and multivariate Cox analysis was used to analyze the variables related to the prognosis of overall survival (OS) and cancer-specific survival (CSS). Propensity score matching (PSM) was used for subgroup analysis of patients with different lymph node status. An overall of 5934 patients was involved with pathologically confirmed thymic malignancies (1975-2016), of which the highest proportion was thymoma (63.3%), followed by TCs (18.5%) and TNETs (5.6%). A total of 1048 TETs individuals underwent surgery and LND. The overall proportion of TETs patients with LNM was 19.1%. The rate of LNM in thymoma, TCs, and TNETs was 6.8%, 30.2%, and 61.1%, respectively. Histology type and T stage were independent factors correlated with LNM in the multivariate Logistics analysis. There were 812 patients with TCs and TNETs underwent surgical treatment, including 76.7% cases of TCs and 11.6% cases of TNETs. About 398 patients underwent LND and 36.2% of patients among them had LNM. In the multivariate Cox analysis of OS and CSS, the prognosis of LND- patients was significantly worse than that of N0 patients (OS: P =0.019; CSS: P =0.012), and the prognostic difference between N+ and LND- patients was not statistically significant (OS: P =0.561, CSS: P =0.759). After PSM, in the univariate analysis and multivariate subgroup analysis of OS and CSS, the survival of N0 patients is still better than that of LND- and N+ groups, however, the prognosis (OS and CSS) difference between LND- and N+ patients did not show statistical significance in multivariable analysis (P >0.05). Nodal involvement was not uncommon in TETs. Main factors related to LNM in TETs were histology type and T stage. LND in TCs and TNETs can achieve a clearer lymph node status and assess the long-period prognosis of patients with more accuracy

A Review of Intrusion Detection Technology Based on Deep Rein-forcement Learning

With the rapid development of modern science and technology, all kinds of network attacks are updated constantly. Therefore, the traditional network security defense mechanism needs to be further improved. Through extensive investigation, this paper presents the latest work of network intrusion detection technology based on deep learning. Firstly, this paper introduces the related concepts of network intrusion detection technology. On this basis, we further evaluate the performance of three common deep learning models in intrusion detection, and conclude that DBN algorithm has some strong advantages. Afterwards, it also puts forward several improvement strategies of intrusion detection models

Unsupervised Deep Feature Transfer for Low Resolution Image Classification

In this paper, we propose a simple while effective unsupervised deep feature transfer algorithm for low resolution image classification. No fine-tuning on convenet filters is required in our method. We use pre-trained convenet to extract features for both high- and low-resolution images, and then feed them into a two-layer feature transfer network for knowledge transfer. A SVM classifier is learned directly using these transferred low resolution features. Our network can be embedded into the state-of-the-art deep neural networks as a plug-in feature enhancement module. It preserves data structures in feature space for high resolution images, and transfers the distinguishing features from a well-structured source domain (high resolution features space) to a not well-organized target domain (low resolution features space). Extensive experiments on VOC2007 test set show that the proposed method achieves significant improvements over the baseline of using feature extraction.Comment: 4 pages, accepted to ICCV19 Workshop and Challenge on Real-World Recognition from Low-Quality Images and Video

Design of actuation system and minimization of sensor configuration for gait event detection for Gen 3.0 Portable Powered Ankle-Foot Orthosis (PPAFO)

Powered ankle-foot orthoses (AFOs), which are capable of providing assistive torque at the ankle joint, have significant potential as both assistance and rehabilitation devices. Technology advancements have led to great progression in the development of powered AFOs. Our group had developed the Portable Powered Ankle-Foot Orthosis (PPAFO) that was capable of providing bidirectional assistive torque at the ankle joint. Two generations of the PPAFO were previously developed. Both designs used two different off-the-shelf rotary actuators. This thesis consists of two studies focusing on the development of a new compact higher torque actuation system and the identification of a minimum sensor configuration for gait event detection for a powered AFO. Study 1 presents the design and evaluation of a new actuation system for the PPAFO (Generation 3.0). The actuation system utilized two dual-action linear actuators and a customized gear train. Compared with the previous designs, it generated higher torque and power while providing a thinner lateral profile. The new design had a total weight of (680g) and was capable of generating 32 Nm torque and 110 W power. While running under the same torque and power level as the previous designs, the new design offered better longevity (42.9% and 81.4% increases in normalized run time for test bench emulation and treadmill walking). Although the overall weight of the new actuation system had a 20% increase compared with previous design, it could generate 166.7% more torque and 120% more power, which will enable us to test the system at various torque and power settings. Study 2 investigated the minimum sensor configuration for detecting gait events. Knowledge of the expected orientation and behavior of a limb as related to specific events during the gait cycle (or state timing as a function of the percentage of the gait cycle, % GC) is essential to allow appropriate control of a powered AFO. A total of five sensors were selected (two force sensitive sensors, one ankle angle sensor, and two inertial measurement units (IMU)). The performances of selected sensor configurations were quantified and compared through state-based and event-based approaches in terms of gait state estimation and gait event detection timing, respectively. Gait data were collected from five healthy subjects while walking on a treadmill wearing the Gen 3.0 PPAFO. Results indicated that, while single IMU configurations (located on the shank or foot) both outperformed all other configurations (mean state estimation error: < 2% GC; mean event detection timing error: < 23 ms), the shank IMU was able to detect more gait events than the foot IMU. Since more detectable events could improve the system's robustness (i.e., adjusting to variable speeds) by updating estimation more frequently, a single shank IMU configuration was recommended for powered AFO applications